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the MALTED system. The work finds that there are serious incompatibilities 
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Abstract: This paper investigates the usability of educational metadata 
schemas with respect to the case of the MALTED (Multimedia Authoring for 
Language Teachers and Educational Developers) project at University College 
London (UCL). The project aims to facilitate authoring of multimedia 
materials for language learning by allowing teachers to share multimedia 
materials for language teaching, and offering a comparatively easy interface 
for retrieving materials and composing exercises and courses from them. 
Although a brief description of the MALTED system is given, the paper 
focuses on usability problems encountered by the project during the 
development of the MALTED system. This work finds that there are serious 
incompatibilities between the current educational metadata schema, as 
promoted by the Learning Object Metadata (LOM) working group of the IEEE 
Learning Technologies Standardization Committee, and the way language 
teachers actually go about authoring teaching materials. 
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1 Introduction 

1.1 Metadata 

Metadata was introduced to facilitate searching for items not only via the items’ technical attributes but 
also via additional information about the items. For example, web pages often contain metadata 
describing the content of the page, enabling search engines to determine the content of the page without 
having to search through the whole page. Consequently, rich metadata schema allows sophisticated or 
very specific search requests to be carried out. 

1.2 Education and Metadata 

The idea of searching for educational materials from external resources and reusing them is not new. 
Education professionals regularly use and reuse materials from outside their classrooms as part of their 
teaching materials, e.g., newspapers, videos, posters and other props. Educational metadata schemas, 
such as the one specified by the Learning Object Metadata (LOM) working group of the IEEE Learning 
Technologies Standardization Committee, aim to help teachers find the right multimedia materials that 
suit their pedagogic needs, by facilitating sharing of those materials among education professionals. LOM 
objectives go as far as to enable exercises to be automatically generated by computer agents that are given 
certain specific criteria. Consequently, LOM metadata schema is extensive. The schema archives a wide 
range of information concerning the materials, ranging from the description of the materials to the kind of 
student interactions the materials are authored for. This not only enables teachers to search for materials 
by subject or a particular course but also specifically by students of certain age, level of competence or the 
kind of interaction the teacher would like the multimedia material to use. 
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Other related metadata projects aim to deal with the practical problems of LOM extensive schema by 
either creating a subset schema that is compatible with the more extensive LOM schema (e.g. ARIADNE) 
or creating an alternative lightweight schema (e.g. Dublin Core Metadata Initiative). 

1.3 The MALTED Proj ect 

MALTED (Multimedia Authoring for Language Teachers and Educational Developers) is a European 
Commission-funded project led by University College London (UCL). The aim of the project is to 
facilitate authoring of multimedia materials for language learning. Computer-based multimedia exercises 
can enrich the self-study part of language learning, which requires a certain amount of drill-style practice, 
for several reasons: 

1. By utilising sound, video and animation, a wider range of language skills can be exercised than 
through paper-based exercises. 

2. Computer-based exercises provide immediate feedback to the learner. 

3. Well-designed multimedia materials appeal to many learners and thus increase the amount of self- 
study. 

For these reasons, many language teachers would like to utilise multimedia exercises. Authoring effective 
and attractive multimedia materials is, however, time-consuming and requires a level of computer skills 
that many teachers do not possess or have time to acquire. Building a time-telling exercise with an 
animated clock and sound clips, for example, can take many hours. MALTED aims to reduce the effort 
required to produce effective and attractive materials by allowing teachers to share multimedia materials 
for language teaching, and offering a comparatively easy interface for retrieving materials and composing 
exercises and courses from them. MALTED collects multimedia materials in a repository for re-use by 
other teachers. To allow teachers to identify suitable materials for a particular exercise, materials 
submitted to the shared repository have to be tagged. Without tagging, identifying suitable materials 
would require tedious and time-consuming scanning of long lists of materials in the repository. Since 
effective and efficient re-use of educational materials is at the core of MALTED, it offered a case study of 
educational metadata in action. 

While this work will outline the technical implementation of the MALTED system (Section 2), the focus 
of the work will be on issues concerning usability of metadata from the point of view of language teachers 
at UCL who are using the MALTED system to author teaching materials for their courses (Section 3). In 
particular, the work will highlight serious incompatibilities between the metadata schema advocated by 
LOM and the processes that language teachers actually employ to author teaching materials (Section 4.1). 
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2 The MALTED System 

2.1 System Architecture of MALTED 




Figure 1: Simplified MALTED System Diagram 

The MALTED system consists of three main parts: MALTED clients, MALTED assetbase and the 
database backend. Figure 1 presents a simplified system diagram of the whole MALTED system. 

The client application provides an interface for teachers to access and manage materials or “assets” and 
their metadata via the MALTED assetbase, allowing them to perform tasks such as searching for and 
adding new assets to the back-end database. It is solely through this client application that teachers will 
interact with the MALTED assetbase. The client application also provides facilities for authoring and 
managing materials. 

The assetbase performs the standardisation of metadata between the client and the database. This is 
accomplished by converting a database query or an update requested by the client in extensible Markup 
Language (XML) into the appropriate Structured Query Language (SQL) query or update for the database 
and vice versa. A custom data dictionary (DD) -- containing a mapping of XML attributes to database 
tables or views, with associated typing - is used together with a Document Type Definition (DTD) to 
enable the assetbase to store the assets and their metadata in the appropriate place on the database. This 
gives the advantage of flexibility as different subsets can be used for different asset types. For example, 
technical information such as resolution is appropriate for a video or image file, but meaningless for a text 
file. The same data dictionary is used to create the XML files to return to the client following a query by 
extracting and combining data from multiple tables, where each XML file represents the metadata 
associated with a single asset. 

A database is used to store assets and their metadata in order to optimise the speed of access for queries 
and to enable complex queries, which may be inefficient if carried out via XML, to be performed. While 
it is possible to use other database packages, the MALTED system uses Oracle, which communicates with 
the assetbase via JDBC (the assetbase having been written in JAVA). 



3 Handling Metadata in MALTED 
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It is possible to achieve a number of LOM objectives directly through the use of metadata to tag 
educational material with relevant information on the material. The following LOM objectives (LOM, 
2000) are particularly relevant when designing a system to handle metadata: 

• To enable learners or instructors to search, evaluate, acquire, and utilize Learning Objects. 

• To enable the sharing and exchange of Learning Objects across any technology supported 
learning systems. 

• To enable the development of learning objects in units that can be combined and decomposed in 
meaningful ways. 

Teachers using MALTED first encounter metadata when searching for assets on assetbases and then while 
contributing new assets to assetbases. The MALTED project has encountered some significant usability 
issues concerning metadata while implementing facilities to support these two tasks. The following issues 
were raised during both formal and informal interviews and focus groups with users of the MALTED 
system. Consequently, for this section, the term "user(s)" will refer to language teacher(s) at UCL, unless 
explicitly stated otherwise. 

3.1 Searching via Metadata 

From the user’s point of view, metadata is first encountered when searching for materials on an assetbase. 
To support this task, MALTED provides a search dialogue in which the user specifies what to search for 
in which metadata field. At this stage, large number of fields in the metadata schema is very useful to the 
user as a large number of fields allow the user to very specifically define a search. For example, the user 
can search for assets for French language teaching at first year undergraduate level with multiple-choice 
questions as the interaction type for students whose mother tongue is not French. Even fields that have 
little or no pedagogic value can be of some use for the user — the fields for file size and format can be of 
use if the user is concerned with download time or compatibility issues. 

After a search request has been submitted, the search result is then displayed to the user showing 
information about the assets structured by the metadata schema. Again, the fields in the metadata schema 
are useful for the user as the schema gives structure to the information on the materials. Flowever, if all 
fields are displayed to the user at the same time, some information that are given to the user may be 
irrelevant and, if there are a large number of irrelevant fields, the user may experience information 
overload. 

3.2 Tagging Material with Metadata 

The rich and extensive metadata schema that helps the user search for and view asset information are, 
however, significant overheads for the user when he/she is adding new material to an assetbase. There are 
a significant number of usability problems concerning metadata that arises when the user performs this 
task. 

3.2.1 Too many fields 

The problem experienced by the user, at this particular stage, is not so much information overload, but 
information "over-demand". Although almost all of the fields are applicable (i.e. the material does have 
attributes which can be categorised by the field), they may not be important. Users feel that it is not 
necessary to identify all possible characteristics of the materials they are submitting to the assetbase, just 
the “relevant” or “important” ones. We found that such overhead imposed on the user could deter the user 
from using the system because it takes too much effort to share materials. Previous research (Davis 1993) 
(Fullan 1982) on the factors influencing implementation of educational technology strongly suggests that 
the user’s perceived practicality of the system is one of the main factors to influence whether the system 
will be used or not. In the case of MALTED, the user was spending a lot of time filling in each field and 
also working out the most appropriate content of each field. MALTED users feel that this process takes 
far too long. 
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3.2.2 Ambiguous field names 

Often the user is presented with two or more fields that, in their opinion, refer to the same attribute of the 
material they are trying to add. For example, users often consider the fields “Content” and “Description” 
to be the same. Consequently, the user finds it difficult to fill in some fields as it is felt that the 
information has already been entered in a previous field. As in the previous problem, the user feels that 
more work than necessary is being demanded of them by the system. 

3.2.3 Irrelevant fields 

Users often do not think about their materials in terms of the fields offered. This is particularly the case 
with users who teaches modem languages. For example, if the user wants to add a picture of a cat to the 
assetbase, the user must specify which language and level of competency the material should be used for. 
While such information on an asset may be appropriate for a whole course, these fields are clearly not 
relevant for such low-level materials such as pictures. Furthermore, such material can be useful for 
authoring lessons or exercises for a wide range of subjects, from modem languages, art and even physics! 
Consequently some fields demand information, which are too specific for certain materials. Ironically, 
highly specific tags may prevent materials from being widely used; contradicting the notion that metadata 
was introduced to allow greater sharing and reuse of materials. 

3.3 MALTED Metadata Interface 

The design of dialogues dealing with metadata in MALTED has to solve the following problems: 

• How to present large number of fields without overwhelming the user? 

• How to get the user to complete the fields as quickly and accurately as possible? 

• How to reduce the number of fields to be filled in by the user? 

• How to determine which fields are relevant for a certain material? 

3.3.1 Automatic Field Completion 

There are a number of fields that can be automatically completed by the system. These are information 
that can be derived from the file system, such as file size, file format and other technical information. 
MALTED automatically fills these fields in for the user when the material is submitted. If the user is 
adding a course to the assetbase, all the assets in that course inherit the tagging from that course. 

3.3.2 Reduction of the number of fields 

MALTED reduces that number of fields for a material by displaying only the fields that are relevant to the 
type of material being added. Taking an earlier example, MALTED will not prompt the user for 
information concerning ’’Image Resolution” if the user is adding a text or an audio file to the assetbase. 
This helps to reduce the problem of having too many and/or irrelevant fields. Additionally, MALTED 
differentiates between mandatory fields and optional fields by hiding optional fields from the user unless 
the user explicitly requests to fill in those fields. 

A possible counter-argument against reducing the number of fields can emerge from what has already 
been discussed in Section 3.1: Searching for Material. Large numbers of fields allow the user to define a 
search very specifically. There will always be a case where a user might want to search for materials 
under an obscure field, such as the "Semantic Density" and the "TaxonPath" of a material (LOM schema). 
However, by attempting to satisfy every possible use of metadata, the metadata schemas such as LOM may 
have made themselves unusable to small-scale or individual users. 



4 Discussion 

While the MALTED project has not been able to solve all the usability problems concerning metadata that 
it has encountered, it has managed to highlight issues that require further debate and enquiry. From a 
broader perspective, during the development of MALTED, knowledge about what functionalities are 
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required by language teachers when authoring electronic multimedia materials has been acquired. This in 
turn has raised the issue of the appropriateness of educational metadata schema in relation to how 
language teachers actually go about creating teaching materials. 

4.1 Material Usage and Metadata 

Problems discussed in Section 3, such as over-demand and overload of both relevant and irrelevant 
information about assets, raise the possibility that sharing and reuse of educational materials between 
users happens at a much lower level of granularity. In the case of MALTED, users just want to pick out 
elements of a course or just find a single diagram that they can use in their own question. The LOM 
metadata schema suggests that users would want to reuse other people’s whole exercises or even courses. 
This raises a fundamental question concerning how teachers and lecturers go about authoring their 
teaching materials and what kind of computer support they actually require. 

4.1.1 Syllabus and Instructional Goals 

The current view of the teacher’s task, in which constructing teaching material is one of adapting an 
existing course created by someone else to suit an area of the syllabus that he/she is trying to cover. The 
LOM metadata schema codifies the course in terms of the syllabus that the course is covering. This 
assumes that what the teacher is trying to accomplish in class - instructional goals - is very similar to the 
syllabus or at least derivable just from the syllabus. However, in the case of language teachers (users of 
MALTED), the task they engage in is a very different one. Instead of finding a course or exercise that 
deals with a certain part of the syllabus and adapting their instructional goals to suit the course/exercise, 
language teachers approach material authoring from the opposite end. That is, they first form a set of 
instructional goals and then find materials - such as pictures, audio or a particular element of an exercise 
(e.g. time-telling elements of an exercise) but not a whole exercise or course - that can fulfil or be adapted 
for each individual instructional goal. The main reason for this “reversed” approach is because language 
teachers feel that they need to know the students’ abilities first before writing instructional goals that 
would be the road map for those students to accomplish what the syllabus have set out. 

While the goal of creating teaching materials by reusing previously authored materials is the same in both 
cases, the sub-goals or how the main goal is achieved are very different. Teachers are not looking for 
“ready-made” courses or exercises to use. Users of MALTED feel that it is their responsibility as teachers 
to identify the instructional goals for their students. Moreover, other people’s exercises may cover the 
right part of the syllabus but not necessary via the most appropriate instructional goals. Supporting the 
task model where the instructional goals are set by someone else would be a critical failure to support the 
correct sub-goals. 



5 Conclusion 

While it was very important to solve technical problems encountered during the implementation of 
MALTED, the issues of the usability of metadata management facilities in MALTED are very significant 
factors in determining whether the system will be used or not by the teaching profession. 

While extensive metadata schema for educational materials facilitates very accurate search queries, the 
schema has serious usability problems from the end-user’s (teacher/lecturer) point of view when 
contributing new materials to an archive. In order to use the extensive LOM metadata schema, a 
professional archive manager will be needed to perform the tagging as the process is too time consuming 
for the teachers to do themselves. MALTED uses a subset of the LOM schema that can be mapped to the 
more extensive schema. 

A significant finding during the MALTED project is that language teachers reuse materials at a much 
lower level of granularity than LOM metadata schema would suggest. Language teachers write 
instructional goals based on the abilities of their students then find materials that would satisfy the 
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pedagogic needs of those instructional goals. Separate assets of a course or exercise, such as pictures or 
audios, are individually picked out by the teachers and are used to author a new course or exercise. 
Consequently, the user requires authoring support at a much lower level of granularity. While the highest 
possible level of granularity of reuse is using a whole pre-authored course, language teachers hardly reuse 
courses by adapting them. They reuse smaller components of courses that are appropriate for instructional 
goals that have been set by the teachers themselves. This model of usage may also be applicable to other 
cases where teachers reuse multimedia materials from resources other than their own. It may also be 
applicable to other subject areas and other levels of education. 
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